A SUPERCRITICAL ELLIPTIC PROBLEM IN A 
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^^ ' Abstract. We consider the problem 

#-s ' ~Au — \u\''~ u in Q, It = on dO,, 

^ ■ where n ■- {{y,z) G E™+i x R^-™-i : < a < |y| < b < cx)}, 

< m < TV - 1 and AT > 2. Let 2*i^^^ := 2{N ~ m)/(N - m - 2) if 
m < TV — 2 and 2J/ „ := oo if m = A — 2 or TV — 1. We show that 
2jv,m is the true critical exponent for this problem, and that there exist 

Q, [ nontrivial solutions if 2 < p < 2^ „j but there are no such solutions if 
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1. Introduction 



> 

CX5 

f^ ■ Consider the Lane-Emden-Fowler problem 

(1.1) -Au=\u\P~^u inP, n = on SP, 

^T . where P is a smooth domain in R-'^ and p > 2. 

O ■ 

^r^ . If P is bounded it is well-known that this problem has at least one positive 

solution and infinitely many sign changing solutions when p is smaller than 

the critical Sobolev exponent 2*, defined as 2* := -^^ if A^ > 3 and as 

^ I 2* := oo if A^ = 1 or 2. In contrast, the existence of solutions for p > 2* 

is a delicate issue. Pohozhaev's identity [12] implies that problem (jl.ip has 

no nontrivial solution if the domain V is strictly starshaped. On the other 

hand, Bahri and Coron [2] proved that a positive solution to (II. ip exists if 

p = 2* and V is bounded and has nontrivial reduced homology with Z/2 

coefficients. 

One may ask whether this last statement is also true for p > 2* . Pas- 

saseo showed in [101 E] that this is not so: for each 1 <?n,<A^ — 2he 

exhibited a bounded smooth domain D which is homotopy equivalent to the 

m-dimensional sphere, in which problem (jl.ip has infinitely many solutions 
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—/S.U = \u\^ u in Jl, 




n = on 9J7, 




^ |Vup,|7x|P€Li(f]), 




:"+! X M^-™-! : a < |y! < 6}, 


< a < 6 < oo 
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\ip <2y^:= jy_~^2 ^^"^ does not have a nontrivial solution if p > 2^^. 
Examples of domains with richer homology were recently given by Clapp, 
Faya and Pistoia in [3]. Wei and Yan established in |17j the existence of 
infinitely many positive solutions for p = 2*^ ^ in some bounded domains. 
For p slightly below 2*^^ solutions concentrating along an ?Ti-dimensional 
manifold were recently obtained in [U H]. Note that 2^ ^ is the critical 
Sobolev exponent in dimension N — m. It is called the (m, + l)-st critical 
exponent for problem (jl.ip . 

The purpose of this note is to exhibit unbounded domains in which this 
problem has the behavior described by Passaseo. 

We consider the problem 

(1.2) 

in a cylindrical shell 
Jl := {x = (y,z) € 

for p > 2. 

If m, = A^— 1 or N—2, we set 2^^ := oo. First note that if m = A^— 1 then 
il = {x G M^ : a < |x| < b}, and a well-known result by Kazdan and Warner 
[9] asserts that (|1.2p has infinitely many radial solutions for any p > 2. In 
the other extreme case, where m = 0, the domain Q is the union of two 
disjoint strips (a, 6) xM and (—6, — a) xM . Each of them is starshaped, 
so there are no solutions for p > 2^q = 2*. Esteban showed in [5] that 
there are infinitely many solutions in (a, 6) x R if A^ > 3 and p < 2*, 
and one positive solution if A^ = 2 (in fact, she considered a more general 
problem). These solutions are axially symmetric, i.e. u{y,z) = u{y, \z\) for 
ah {y,z) E Q. 

Here we study the remaining cases, i.e., 1 < ?n, < A^ — 2. Our first result 
states the nonexistence of solutions other than n = 0, if p > 2^^. 

Theorem 1.1. Ifl<m<N — 2 and p > 2^^, then problem (II. 2j) does 
not have any nontrivial solution u € C'^{0,) nC^{Q). 

Our next result shows that solutions u ^ do exist if 2 < p < 2^^. 

As usual, we write 0{k) for the group of linear isometrics of W^ (repre- 
sented by orthogonal k x /c-matrices). Recall that if G is a closed subgroup 
of 0{N) then a subset X of M^ is G- invariant if gX = X for every g ^ G, 
and a function n : X — )■ M is called G-invariant provided u{gx) = u{x) for 
ah 5 G G, X e X. 
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Note that Q is [0{m + 1) x 0{N — m — l)]-invariant for the obvious ac- 
tion given by ((7, h){y, z) := {gy, hz) for all (7 G 0(m + l), /i € 0{N — m— 1), 



Theorem 1.2. (i) If I <m< N -2 and 2 < p < 2|^^, then problem (II. 2p 
has infinitely many [0{m + 1) x 0{N — m — l)]-invariant solutions and one 
of these solutions is positive. 

(a) Ifl<m = N — 2 and 2 < p < 00, then problem (jl.2p has a positive 
[0{N — 1) X 0{l)]-invariant solution. 

In Section [2] we prove Theorem 11.11 Theorem 11.21 is proved in Section [3l 
We conclude the paper with a multiplicity result and an open question in 
Section HI 

2. A POHOZAEV IDENTITY AND THE PROOF OF THEOREM 11.11 

We prove Theorem 11.11 by adapting Passaseo's argument in [lOlIll], see 
also [3]. The proof relies on the following special case of a Pohozaev type 
identity due to Pucci and Serrin [13] . 

For (n, u) € M X R^ we set 

J./- \ 1 I |2 ^ I IP 

(piu, V) := - \v\ \u\'^ . 

2 p 

Lemma 2.1. If u £ C'^{Q,) satisfies —Au = \u\^^ u in U then, for every 
X G C^(n,M^), the equality 

(2.1) (divx) Hu, Vu) - Dx [Vu] • Vu = div [</)(«, Vu)x - (x • Vu)Vn] 

holds true. 

Proof. Put X = (xii • • • jXn), denote the partial derivative with respect to 
Xk by dk and let LHS and RHS denote the left- and the right-hand side of 
([2T]1 . Then 

LHS = (divx) </'('U, Vu) — >^ dkXj djudkU 

j,k 
and 

RHS = (divx) Hu, Vu) + ^Xk djud^u - \u\P-^uVu ■ x 
- (Vu • x) A-u - ^ dkXj dju dkU - ^ Xj dkU dj,,u 

j,k j,k 

= {di-vx)4>{u,Vu) - (Vu-x)(An+ |njP"\) -^dkXj djudkU. 

j,k 

Since —Au = |ti|^^ u, the conclusion follows. D 
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Using a well-known truncation argument, we can now prove the following 
result. 

Proposition 2.2. Assume that x ^ C"'^(ri,M ) has the following properties: 

(a) X'^ is bounded on dil., where ^{s) is the outer unit normal at s £ dQ, 

(b) |x(2^)l ^ l^^l for every x G fi, 

(c) divx is bounded in il,, 



(d) \Dx{x)^ ■ ^\ < \Cr for all X e n, U 



oTV 



Then every solution u E C'^{^) nC^(J7) of (|1.2p satisfies 



(2.2) 



^ Jan 



I Vu| X • ^ 



/ (divx)</)(u,Vu)+ / L'xlVnJ-Vu. 



Proof Choose V G C~(M) such that < V(i) < 1, V'(i) 
V'(t) = if |t| > 2. For each k e'M define 

/l |2\ 



1 if |t| < 1 and 



ipkix) := V 



A;2 



and 



^''(a;) := Tpk{x)xix). 



Note that there is a constant cq > such that 

(2.3) k||VV'fc(x)| < Co for aU X € M^, /c e N. 

Next, choose a sequence of bounded smooth domains ^2^ C fi such that 

(2.4) f]fcDr?nS2fc(0). 

Integrating (|2.ip with x := x^ in ^2,^ and using the divergence theorem and 
Lemma |2. II we obtain 

/ fdivx'') Hu, Vu) - / Dx'' [Vu] • Vu = 

f U(n, Vu) (x" ■ ^') - ix" ■ Vu) ( Vu • i/^) 1 , 

where u^ is the outer unit normal to rj^. Property (j2.4p implies that x'^ = 
in fi \ Qfc, so we may replace O^ by $7, d^k by 90 and z^ by u in the 
previous identity. Moreover, since u = on d^, we have that 

Vu = (Vu • z^) 1/ on 9ri. 

Therefore, 



(2.5) 



an 



9f7 



divx'') Hu, Vu) - / Dx'' [Vu] • Vu = 
(u,Vu)fx'^-i/)-(x'^-Vu)(Vu-i/) 



(u, Vu) 



iVul 



X''-!^ 



1 



an 



|VupVfc(2^)(x- i^) 
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Since div x^ = V'fc div x + ^i^k ■ X; using (12. 3p and properties (b) and (c) we 
obtain 



(2.6) 



divx 



< |divx| + |V^fc[ Ixl < |divxi + cq < ci in 0. 
Similarly, since 

DxH^)^ • ? = M^)Dx{x)^ ■ ? + (VV-fe • (X • , 
property (d) yields 



(2.7) 



Dx''{x)C • C < (1 + Co) JCP for all X G f^, C € 



cAf 



Inequalities (|2.6p . (j2.7p and property (a) allow us to apply Lebesgue's dom- 
inated convergence theorem to the left- and the right-hand side of (j2.5p to 
obtain 

1 



(div x) Hu, V«) - / Dx [Vn] ■Vu = -- I | Vn|" (x ■ i^) , 
n Jn ^ Jan 



as claimed. 



D 



Proof of TheoremllJl Let ip{t) = ^ [l - (f )™+^] be the solution to the 
boundary value problem 

r f'{t)t + (m + l)(p{t) = 1, te (0, oo), 
I via) = 0. 

Define 

(2.8) xiy,z):=iip{\y\)y,z). 

Then, if u denotes the outer unit normal on di}, 

(2.9) (x-)(y,.) = {ii_ [!_(.).+!], t\l\zl: 

So property (a) of Proposition [2^2] holds. Clearly, (b) holds. Now, 

(2.10) divx(y,z) = [ip'{\y\)\y\ + {m + l)ip{\y\)] +N-m-l = N-m. 
In particular, (c) holds. To prove (d) notice that x is 0{m + l)-equivariant, 



I.e. 



Xiay, z) = gx{y, z) for every g G 0{m + 1). 
Therefore, g o Dx{y, z) = Dx{gy, z) o g and, hence, 

{Dx {y, z) [i] ,0 = {9 {Dx {y, z) [i]) , gi) = {Dx {gy, z) [gilgi) 

for all ^ G M^. Thus, it suffices to show that the inequality (d) holds for 
y = (t, 0, . . . , 0) with t G (a, 6). A straightforward computation shows that, 
for such y, Dx{y) is a diagonal matrix whose diagonal entries are an = 
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1 — 'nnp{t), ajj = ip{t) for j = 2, . . . , m + 1, and Ujj = 1 ior j = m + 2, . . . , N . 
Since ajj G (0, 1], 

(2.11) 0<{Dx{y,z)[C],O<\^\^ for alU G M^ \ {0} 

and (d) follows. From I^M), (122]), (l2TT]) and (|230]) we obtain 

0<^ / \Vu\\-i^ = - I (divx)0(n,Vn)+ [ Dx[Vu]-Vu 
^ Jdn Jn Jn 

|2 



1 I IP 1 IY7 |2 

- \u\ Vu 

p' ' 2 ' ' 



+ / |Vu|' 



11 1 \ , ._ ,2 



<{N-m) 

= (N -m){ - - - + —^ — ) / [V-u 

The first (strict) inequality follows from the unique continuation property 
[H E]. This immediately implies that p < 2^^. D 

3. The proof of Theorem 11.21 

An 0{m + l)-invariant function u{y, z) = v{\y\ , z) solves problem ()1.2p if 
and only \i v = v{r, z) solves 

r -At>-^|^ = |z;|P-2t; in(a,6)xM^— 1=:5, 
^ ' \ v = Q on {a, 6} x M^-™-i = dS, 

and |Vf p, \v\P G L^(5). Problem (|3.ip can be rewritten as 

(3.2) -diY{r'^Vv)=r'^\v\P-'^v in 5, v = on 95. 

By Poincare's inequality (see Lemma 3 in [5j) and since a < r < b, the 
norms 

1/2 / r \ Vp 

lm,p 



(3.3) ||t;|L:= ( / r^lV^n and \v\ 



IS 

are equivalent to those of Hq{S) and ]J'{S) respectively. 
Consider the functional I{v) := \\v\\^ restricted to 

M:={v(^Hl{S):\v\^^^ = l}. 

Then M is a C^-manifold, and u is a critical point of I\m if and only if 
V G Hq{S) and ||i;||m v is a nontrivial solution to ()3.2p . Note that /|m 
is bounded below by a positive constant. 

Proof of TheoremlTE (i)- Assume that 1 < m < A^ - 2 and 2 < p < 2^„^. 
Set G := 0{N -m-l) and denote by H^{S)'^ and LP{S)^ the subspaces of 
Hq{S) and LP{S) respectively, consisting of functions v such that v{r,gz) = 
v{r, z) for all g ^ G. Esteban and Lions showed in [6j that, for these values 
of m and p, Hq{S)^ is compactly embedded in L'p{S)'^ (see also Theorem 
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1.24 in [IB]). So Hq{S) is compactly embedded in L^^S) for the norms 
(j3.3p as well. 

Let 

MG:={v€H',iSf:\v\^^p = l}. 
It follows from the principle of symmetric criticality |181 Theorem 1.28] that 
the critical points of I\mg ^^re also critical points of I\m- The manifold M 
is radially diffeomorphic to the unit sphere in Hq{S) , so its Krasnoselskii 
genus is infinite. A standard argument, using the compactness of the em- 
bedding Hq{S)'^ "-^ LP{S)^ for the norms (|3.3p . shows that I\mg satisfies 
the Palais-Smale condition. Hence I\mg has infinitely many critical points 
(see e.g. Theorem II. 5. 7 in [I5])- It can also be shown by a well-known ar- 
gument that the critical values of I\mg tend to infinity (see e.g. Proposition 
9.33 in m). 

It remains to show that (13. 2p has a positive solution. The argument is 
again standard: since I\mg satisfies the Palais-Smale condition, 

c^ := mf{I{v) : v G M^} 

is attained at some vq. Since I{v) = I(\v\) and \v\ S M*-' if i; € M^, we 
have that /(|fo|) = Cq and we may assume vq > 0. The maximum principle 
applied to the corresponding solution uq of (|1.2p implies uq > 0. D 

If m = A^ — 2, then G = 0(1) and it is easy to see that the space Hq{S)^ 
is not compactly embedded in LP{S)^ . So part (ii) of Theorem [L2] requires 
a different argument. 

Proof of Theorem \1.2\ (ii). Assume that 1 < m = N — 2 and 2 < p < oo. 
We shall show that 

Co := inf{/(z;) : v e M} 

is attained. Clearly, a minimizing sequence (vn) is bounded, so we may 
assume that Vn ^ v weakly in Hq{S). According to P.-L. Lions' lemma \1S\ 
Lemma 1.21] either u„ — > strongly in LP{S), which is impossible because 
Vn S M, or there exist 5 > and (r„, Zn) € [a, 6] x R such that, after passing 
to a subsequence if necessary, 

(3.4) / vl>6. 



f vl 

JBi{r„,z„.) 



Here -Bi(r„, z„) denotes the ball of radius 1 and center at (r„, z„). Since the 
problem is invariant with respect to translations along the z-axis, replacing 
Vn{r,z) by Vn{r,z + Zn), we may assume the center of the ball above is 
(r„,0). It follows that for this - translated - sequence the weak limit v 
cannot be zero due to p.4p and the compactness of the embedding of Hq{S) 
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in Lf^^{S). Passing to a subsequence once more, we have that Vn{x) -^ v{x) 
a.e. It follows from the Brezis-Lieb lemma |18l Lemma 1.32] that 

1 = Pnlm.p ~ -^™ Pn ~ '^Im.p ' \'^\m,p- 

Using this identity and the definition of cq we obtain 

Co = lim ||ti„||^ = lim Hun - f ||^ + ||i;||^ > cq ( lim \vn - v\l^ + \v\'^ 

= CO ((1 - IvL,,)'/^ + {\v\?^,,r^^) > co(l - l^^l^,, + |H^,p)'/^ = CO. 

Since u 7^ 0, it follows that 1^™ — v\m,p -^ and \v\m,p = 1- So u E M and, 
as Cq = lim„^oo I{vn) > -^(^')) we must have I{v) = cq. 

So the infimum is attained at v and using the moving plane method [18^ 
Appendix C], we may assume, after translation, that v{r,—z) = v{r,z), i.e. 
V E Hq{S)^^^' . As in the preceding proof, replacing v by \v\, we obtain a 
positive solution. D 

4. Further solutions and an open question 

Ifl<m, = A^ — 2 and p £ (2,2^^), the method we have used to prove 
Theorem 11.21 only guarantees the existence of two solutions to problem ()1.2p . 
one positive and one negative, up to translations along the z-axis. However, 
if p G (2,2*), then it is possible to show that there are infinitely many 
solutions, which are not radial in y, but have other prescribed symmetry 
properties. 

Write y = (y\ y^) G M^ x W"-^ = M^+i and identify M^ with the complex 
plane C Following [16], we denote by Gk, k > 3, the subgroup of 0(2) 
generated by two elements a, /3 which act on C by 

ay' := e'-^/'y\ /3y^ := e'^^/'^, 

i.e. a is the rotation in C by the angle In/k and (3 is the reflection in the 
line 2/2 = tan(7r/A;)y}, where y' = yl + 'iy2 G C. Observe that a,/3 satisfy 
the relations a^ = /3^ = e, a/3a = a. Let G^ act on M^ by gx = {gy' ,y'^, z). 

Theorem 4.1. Ifl<m<N — 2 and 2 < p < 2* then, for each k > 3, 
problem (jl.2p has a solution Uk which satisfies 



(4.1) Uk{x) = det{g)uk{g 'x) for all g G Gfc, 

and Uk 7^ Uj if k ^ j. 

Proof. Since the approach is taken from [16j , we give only a brief sketch of 
the proof here and refer to Section 2 of [16j for more details. 
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The group Gk acts on Hq{0) by 

{gu){x) := dei{g)u{g''^x), 

where dei{g) is the determinant of g. Let 

Hl{n)^^ := {u G Hl{Q) : u{gx) = det{g)u{g-^ x) for all g G Gk} 

be the fixed point space of this action, and define I{u) := f^ | Vn| and 

M^^ := {u G H^inf'^ : \u\p = 1}. 

By the principle of symmetric criticality the critical points of -^I^/g^ are 
nontrivial solutions to problem (II. 2p which satisfy (14. 1|) . Now we can see 
as in the proof of part (ii) of Theorem 11.21 that there exists a minimizer Uk 
for / on the manifold M^'' . Moreover, we may assume that Uk has exactly 
2k nodal domains, see Corollary 2.7 in [16]. So in particular, Uk 7^ Uj if 
k^j. D 



The question whether problem ()1.2p has infinitely many solutions when 
1 <m = N — 2 and p G [2*, 2^ ^) remains open. We believe that the answer 
is yes, but the proof would require different methods. 
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